May I submit to you an alternative route? For urbs, I alsow wanted to speed up storing the result, while not having the user to maintain pre-knowledge of what they wanted to know from the result later. For that, I collected thoughts in issue tum-ens/urbs#47 and implemented a generic result cache generator in PR tum-ens/urbs#117.

Could you have a look at that and decide whether you like that idea, too? If not, I would be fine with merging this. I just thought that you might find the other design (almost no user-facing changes in "contract" with what the model object contains, except for a missing model after a reload from a HDF5 file) applicable here, too.

Thank you for the suggestion. I will have a look on the urbs solution.

HDF5

If I remember correctly MatLab uses HDF5, so I suppose that is a method worth considering.
(I suppose, the added "cache" is not that big to have a big effect during run-time, right?)
Have you made some benchmarks, to see the difference between pickle-gzip-all versus the HDF5 version? (Time and size difference would be interesting especially)

Modelless pickle

For transparency, you altered get_constants() to handle the _result "cache".
I could see that happening with the proposed minimal save too.
Then hdf5 conversion is not needed, and the function calls remain the same.
Of course, this implicates that the user is somehow expecting the minimal data.
For your concerns regarding the user:

• What about a simple warning/info during load? (At save the user already has to know what he is doing to achieve minimal export)
• Or simply duck-typing (as advocated in your other code-review) could reveal possible incompatibilities if someone tries to use a data-prob as a pyomo-prob.

• Anyhow, I see that this is handling a sub problem, as it does not solve the case if someone does want to store the whole model
  • I already thought about this, and wanted to ask, whether you already profiled the create_model() function?
    EDIT: Speeding up model creation #26 happened since this comment. Model creation is not so slow as used to be. Makes the following question more legit in my opinion.
  • Or what is the sense of storing the whole model? Why would somebody load and then re-run it instead of recreating the model and run it, if he already has the input data + results and does not want to modify the input parameters?